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RESEARCH  SUMMARY 


Abstract 

High  Performance  Computing  (HPC)  is  an  enabling  technology  for  a  large  class  of 
Army  needs  including  weapon  system  design,  simulation  and  modeling,  intelligence, 
automation,  advanced  manufacturing,  and  training.  Recognizing  this,  in  1989  Army 
established  the  Army  HPC  Research  Center.  The  Center  consists  of  four  integrated 
activities:  interdisciplinary  research  into  various  aspects  of  HPC  including  novel  solution 
techniques,  advanced  algorithms,  applications,  and  graphics  and  visualization  (GV); 
evaluation  of  advanced  computing  systems  and  implementation  of  an  advanced  HPC 
environment;  infrastructure  support  and  technology  transfer  to  Army  and  other  DoD 
activities;  and,  an  aggressive  outreach  program  through  participation  of  Historically  Black 
Colleges  and  Universities  (HBCU)  and  Minority  Institutions.  The  prime  contractor  for  the 
Center  is  the  University  of  Minnesota;  its  HBCU  partners  are  Clark  Atlanta,  Florida  A&M, 
Howard,  and  Jackson  State  Universities;  and  its  industrial  partner,  the  Minnesota 
Supercomputer  Center,  Inc.  (MSCI),  maintains  and  operates  the  supercomputing  resources 
of  the  Army  HPC  Research  Center. 

Research  Activities 

A  number  of  Center  interdisciplinary  research  teams  worked  on  HPC  strategies  using 
advanced  computing  systems,  and  applied  these  powerful  strategies  to  Army  problems. 
The  Simulation  and  Modeling  Team  focused  on  techniques  to  provide  solutions  to 
complex,  3D  problems.  Finite  element  computation  of  compressible  and  incompressible 
flows  involving  complex  geometries  with  moving  boundaries  and  interfaces  is  one  of  the 
major  Center  activities.  In  this  area,  ARL  and  Center  researchers  are  collaborating  on  the 
regenerative  liquid  propellant  gun  (RLPG)  and  on  missile  aerodynamics.  In  a  collaborative 
effort  with  Natick  RDEC,  the  Center  is  simulating  the  deployment  and  gliding  of  large  ram- 
air  parachutes. 

The  Advanced  Manufacturing  Team  focused  on  HPC  techniques  and  software  for 
advanced  material  design  and  manufacturing  processes.  The  advanced  material  design 
efforts  emphasize  simulation  of  resin  transfer  molding  and  include  collaborations  with  ARL 
and  Center  for  Composite  Materials  (CCM)  at  University  of  Delaware.  The  efforts  in 
manufacturing  processes  emphasize  computer  aided  manufacturing,  including  process 
planning  for  on-demand  manufacture,  and  algorithms  and  software  for  vision,  robotics, 
and  adaptive  control. 

The  Environmental  Sciences  Team  is  working  with  the  CEWES  in  application  of 
advanced  HPC  techniques  to  groundwater  modeling  and  environmental  fluid  mechanics. 
These  activities  include  developing  new  models  to  predict  the  dispersion,  absorption, 
capillary  pressure,  and  dissolution  of  contaminants  in  groundwater,  as  well  as  developing 
new  computational  hydrodynamics  tools  based  on  the  Navier-Stokes  equations  and  their 
shallow-water  approximations. 

The  Information  Technology  and  Algorithms  and  Software  Technology  Teams  are 
focusing  on  tools  which  facilitate  efficient  and  easy  implementation  of  many  applications  on 
various  advanced  HPC  architectures.  These  tools  include  parallel  scalable  algorithms  and 
libraries;  database  support  for  manufacturing  and  simulation;  and  virtual  interactive 
distributed  simulation.  These  teams  also  focus  on  visualization  of  large  data  sets  resulting 
from  simulations  based  on  structured  and  unstructured  computational  grids.  In  a  joint  effort 
with  TARDEC,  the  Center  researchers  are  also  working  on  numerical  methods  and 
software  for  real-time  simulation  of  multibody  systems  and  interactions  between  rigid-body 
systems  and  deformable  systems. 
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Advanced  HPC  Environments 

As  a  result  of  an  aggressive  acquisition  and  systems  integration  program  as  well  as 
being  able  to  capitalize  on  the  computing  environment  extant  at  the  MSCI,  Army  Center 
researchers  have  access  to  a  feature-rich,  robust,  heterogeneous  computing  environment 
that  is  a  model  for  excellence  in  HPC.  The  hardware  provided  by  the  Army  includes  a 
Thinking  Machines  CM-5  with  896  processing  nodes  and  an  advanced  GV  lab,  whereas 
the  MSCI  computational  resources  include  a  Cray  T3D  with  128  processing  nodes,  a  Cray 
Y-MP  C90  with  9  processors,  and  a  Cray-2  with  4  processors.  This  integrated 
environment  provides  researchers  with  the  latest  in  computing  technology,  high-speed 
networking,  and  user-friendly  system  and  graphics  utilities. 

The  Army  Center  researchers  have  always  been  at  the  forefront  of  effective  and  relevant 
utilization  of  advanced  HPC  systems.  Soon  after  the  CM-5  supercomputers  became 
available  nationwide,  the  Center  researchers  were  among  the  first  to  carry  out  application 
computations  on  this  platform.  The  Center  researchers  were  also  among  the  first  who 
carried  out  application  computations  on  the  Cray  T3D.  At  Supercomputing  '92,  the  Center 
used  heterogeneous  computing  in  simulating  thermal  flows  in  the  Earth's  mantle.  The 
application  executed  on  four  different  HPC  platforms:  a  CM-5,  a  Cray-2,  a  CM-200,  and 
an  SGI  workstation.  This  demonstration  was  judged  the  "Best  of  the  Best"  in 
Supercomputing  '92  Heterogeneous  Challenge  Competition.  In  another  heterogeneous 
computing  application,  recently  the  Center  researchers  simulated  the  flow  inside  a  RLPG 
by  using  the  Cray  C90  at  MSCI  for  the  direct  solution  of  the  coupled  equations,  while 
using  the  Center's  CM-5  for  the  rest  of  the  computations.  The  exchange  of  data  every  time 
step  was  accomplished  over  a  HIPPI  channel. 

The  GV  Laboratory  at  the  Center  is  a  unique  facility  specifically  to  support  the 
demanding  requirements  of  visualization  of  gigabyte-size  data  sets.  With  the  software 
written  mostly  at  the  Center,  the  researchers  were  able  to  visualize  the  3D  flow  simulation 
data  generated  by  using  structured  grids  as  well  as  unstructured  grids  on  complicated 
geometries. 

Software  Development 

The  Center  supported  efforts  to  develop  systems  software,  mathematical  subroutine 
libraries,  GV  tools,  and  applications  software.  These  were  made  available  to  the  broader 
Army  and  HPC  community  through  on-line  libraries  and  one-on-one  research 
collaborations.  A  good  example  of  the  system  software  developed  at  the  Center  is 
Distributed  Job  Manager  which  provides  interactive  and  optimal  use  of  parallel  computer 
resources. 

In  the  category  of  GV  software,  Brick-of-Bytes  (BOB)  has  been  very  widely 
disseminated  to  government,  academe  and  industry.  For  example,  BOB  is  extensively  used 
by  researchers  in  their  modeling  of  composite  materials  at  the  CCM.  BOB  provides  for 
interactive  volume  rendering,  and  efficiently  visualizes  very  large  3D  data  sets. 

DASPK,  a  mathematical  subroutine  library  of  iterative  algorithms  for  solving 
differential  algebraic  equation  systems,  and  its  data-parallel  and  message-passing-parallel 
versions  found  widespread  acceptance  at  many  research  sites  across  the  country  and  is  used 
extensively  at  TARDEC. 

Application  software  developed  at  the  Center  have  also  proven  to  be  very  useful  to  the 
Army.  For  example,  about  the  software  developed  for  the  flow  simulation  inside  a  RLPG, 
an  Army  official  commented  that  this  software  "...provided  crucial  analysis  in  a  short  time- 
frame  which  was  instrumental  in  the  rapid  recovery  of  our  weapon  to  the  firing  line.". 
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Technology  Transfer,  Training  and  Collaboration 

The  Center  has  a  major  commitment  to  technology  transfer,  training  and  collaboration. 
As  an  innovative  computational  testbed,  it  is  the  focus  of  several  technology  transfer 
projects  designed  to  move  university  research  directly  into  Army  labs  and  research  centers. 
Several  infrastructure  support  experts  placed  at  both  the  Center  and  the  Army  labs  played  a 
fvital  role  in  the  day-to-day  operation  of  the  Center  and  were  an  important  link  to  the 
research  program,  serving  as  a  bridge  between  the  university  and  Army  researchers. 
Eaxmples  of  technology  transfer  and  collaborative  projects  with  the  Army  sites  are: 

•  Real-time  simulation  of  large-scale  multibody  systems  (TARDEC) 

•  Design  of  liquid  propellant  guns  (ARL) 

•  Groundwater  modeling  (CEWES) 

•  Parafoil  aerodynamics  (Natick  RDEC) 

•  Resin  transfer  molding  (ARL) 

The  Center  researchers  have  also  been  involved  in  industrial  collaborations  in  areas 
such  as  hardware  and  software  development,  materials  processing,  environmental  fluid 
mechanics,  and  flow  simulation  of  high-speed  transportation  vehicles. 

International  collaborations  were  also  encouraged.  Examples  include  collaborations 
with  the  Chuo  University  in  Japan;  Observatoire  de  la  Cote  d'Azur  in  France;  and  Ecole 
Centrale  de  Lyon  in  France. 

Education,  Training  and  Outreach 

Many  Ph.  D.,  M.  S.,  and  undergraduate  students  were  involved  in  the  Center  research 
projects.  Approximately  20-25  graduate  students  and  10-15  postdoctoral  fellows  were 
funded  each  year  by  the  Center.  Additional  graduate  students  and  postdoctoral  fellows  were 
funded  by  other  federal,  state  and  industrial  research  grants  leveraged  by  the  resources  and 
activities  at  the  Center.  For  example,  three  of  the  postdoctoral  fellows  originally  funded  by 
the  Center  were  also  awarded  Postdoctoral  Associateships  in  Computational  Science  and 
Engineering  by  the  National  Science  Foundation.  Graduate  students  were  encouraged  to 
spend  time  at  the  Army  labs,  working  on  projects  supervised  jointly  by  the  Army  and 
Center  researchers.  A  Center  graduate  student  who  spent  several  months  at  the  ARL 
working  on  numerical  simulation  of  liquid  propellant  guns  later  accepted  a  postdoctoral 
appointment  there. 

Since  the  Summer  of  1991,  the  Center  has  organized  an  annual  six- week  summer 
institute  for  training  of  promising  undergraduate  students.  Each  summer,  15-20  students 
from  the  United  States  and  Puerto  Rico  attended  this  intensive  training  in  HPC,  with 
emphasis  on  numerical  methods,  parallel  computing,  and  GV.  The  program  also 
encouraged  these  students,  especially  women  and  minorities,  to  pursue  graduate  studies  or 
careers  in  HPC.  The  Center  faculty,  postdoctoral  fellows,  and  graduate  students  served  as 
lecturers  and  project  mentors  for  the  students.  The  Army  researchers  were  also  invited  to 
give  lectures,  providing  information  on  the  research  activities  at  the  Army  labs  and  potential 
internship  and  career  opportunities.  The  1994  Summer  Institute  had  18  students  from  13 
institutions  including  Clark  Atlanta,  Florida  A&M,  Howard,  and  Jackson  State 
Universities. 
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